A Preliminary Study of Disputation Behavior in Online Debating Forum

In this paper, we propose a task for quality evaluation of disputing argument. In order to understand the disputation behavior, we propose three sub-tasks, detecting disagreement hierarchy, refutation method and argumentation strategy respectively. We first manually labeled a real dataset collected from an online debating forum. The dataset includes 45 disputing argument pairs. The annotation scheme is developed by three NLP researchers via annotating all the argument pairs in the dataset. Two under-graduate students are then trained to annotate the same dataset. We report annotation results from both groups. Then, another larger dataset was annotated and we show analysis of the correlation between disputing quality and different disputation behaviors.